Evaluating Judges

Much of the interest in empirical studies of judges lies in the comparison of actual to ideal behavior. When we ask what makes a good judge or whether a judge rightly decided a case, we implicitly compare the judge's decisions to a normative standard. In some instances, the content of the normative standard is uncontroversial and its application straightforward. Hence, a trial judge who sentences Black criminal defendants to longer terms of imprisonment than White criminal defendants, all other things being equal, departs from the normative ideal that application of criminal laws should be color-blind 1983). Or the circuit judge who votes to uphold a state law banning abortions on grounds that Roe v. Wade was wrongly decided departs from the normative ideal that inferior court judges should follow the Supreme Court's constitutional decisions (for evidence on how circuit judges view their obligations within the legal hierarchy, see Klein, 2002). In many instances, however, the content of the normative standard chosen for comparison purposes proves controversial or hard to delimit and its application unclear. Is the judge who applies a rule of evidence according to its plain language a " better " judge than one who applies the rule to achieve its purpose when following the text would contravene the purpose? Is the judge who deviates from the law to correct a disparate impact the law is having on minorities, say, in sentencing, a bad judge? Is the circuit judge who narrowly construes the Supreme Court's abortion cases a better judge than the circuit judge who broadly construes this precedent? Does it matter if the former judge favors this minimalist approach due to pragmatic commitments rather than political preferences? Is there even a neutral approach to construal of Supreme Court precedent that can be labeled the way of the ideal judge? 2 Any empirical study that attempts to address these or other questions bearing on the competence of judges confronts three basic tasks: (1) specifying a defensible normative standard as the baseline for comparison; (2) converting the standard into testable form and judicial behavior into measurable units; (3) interpreting the results of any comparison to draw appropriate conclusions about the descriptive-normative gap. 1 I consider complications at each stage in the comparison process, with illustrations from existing studies of judicial competence and behavioral decision theory studies from psychology, which examine the gap between actual judgment and decision-making behavior and norms …